Method and apparatus for measuring dimensions of objects on a conveyor

ABSTRACT

A method and apparatus for measuring the length, width, and height of rectangular solid objects moving on a conveyor. The apparatus includes a light curtain, two laser triangulation range finders, and a pulse tachometer mounted on a frame around a conveyor. As an object is conveyed through the frame, measurements from each of the sensors are correlated by a digital computer to estimate the length, width and height of the object. The method and apparatus does not require a break in the surface of the conveyor, and is insensitive to object placement and orientation on the conveyor.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates to a method and apparatus for measuring the sizeof objects on a moving conveyor. More specifically, the inventionrelates to a method and apparatus for measuring the size of objects on aconveyor using a photodiode light curtain to measure the height of theobjects together with a two laser rangefinders which employ lightdetection cameras and pulse tachometers to measure the length and widthof the objects on the conveyor.

2. Description of the Related Art

Objects traditionally have been measured by hand with ruledstraight-edges and tape measures. However, in a conveyor environment, inwhich the objects on the conveyor are moving (i.e., excess of 100objects per minute passing through a particular area on the conveyor),this is an unreasonable method for determining the size of the objects.

Unger et al. (U.S. Pat. No. 3,436,968 and U.S. Pat. No. 3,588,480)describes a dual light curtain system with one set of beams positionedhorizontally with respect to the conveyor, and another set of beamspositioned vertically with respect to the conveyor. Such an arrangementof a two-dimensional light curtain requires a break in the surface ofthe conveyor, due to the photodetectors on the bottom edge of the lightcurtain that interfere with the conveyor. The Unger system isundesirable due to the needed break in the surface of the conveyor,which is not always available in existing conveyor systems.Additionally, the Unger system requires carton orientation alignmentprior to the measurement by the two-dimensional light curtain.

Henderson et al. (U.S. Pat. No. 3,513,444) describes a dual lightcurtain system which accumulates the sum of volume slices as the objectis conveyed. This system can only measure the volume, and requirescarton orientation alignment, as well as requiring a break in thesurface of the conveyor.

Claesson et al. (U.S. Pat. No. 4,773,029) describes the use of two lightcurtains, one with horizontal beams and one with vertical beams, tomeasure objects on a conveyor. Like the above-mentioned system, theClaesson system also requires a break in the surface of the conveyor.

Hayashi et. al. (U.S. Pat. No. 4,905,512) describes a method similar tothe dual light curtain system of Claesson, except that in the place ofan array of sensors, a single sensor is moved back and forth. The timerequired to move the sensor back and forth strongly limits the speed atwhich objects can be moved on the conveyor. This makes the Hayashisystem inappropriate for high speed object measurements. The Hayashisystem is appropriate for measuring long, continuous materials thatchange little in their respective lateral dimensions over time, such asextruded dough in a baking process.

Stringer et al. (U.S. Pat. No. 5,105,392) describes several measurementschemes, only one of which measures objects in motion. The in-motionscheme uses multiple ultrasonic sensors to sense the top, sides and backof an object when the front of the objects hits a cross-conveyorphoto-eye. The use of ultrasonics carries with it some difficulties,such as the changes in the speed of sound due to temperature and airpressure, the relatively slow repetition rate due to the travel time ofsound, and the large cone of solid angle sampled by the sensor, whichmakes it difficult to ascertain which part of an object is beingmeasured. The Stringer system also requires that the object beorientationally aligned.

Stringer et al. (U.S. Pat. No. 5,220,536) describes a light curtain withvertical beams to obtain width information and ultrasonic sensors to getthe height information. This approach suffers from the temperature andpressure sensitivity and repetition rate limitations of ultrasonics, aswell as the vertical light beams requiring a break in the surface of theconveyor.

Jenseen et al. (U.S. Pat. No. 5,331,118) describes a system similar tothe dual light curtain approaches described above, but utilizes barcodescanners and an array of barcodes in place of the light curtains. Thisapproach suffers from the added complexity associated with thenon-orthogonal geometry of the measuring scheme, as well as requiring abreak in the surface of the conveyor.

Sjodin et al. (U.S. Pat. No. 4,179,707) describes a measuring system forsubstantially parallel objects on a conveyor using a light source, abackground area for receiving light reflected of the objects, and acamera for receiving the light reflected off the background area. Thissystem suffers from the need to have the objects substantially alignedprior to the measurement process, as well as requiring a backgroundmedium for the camera to properly receive the reflected light therefrom.

Dreyfus et al. (U.S. Pat. No. 4,226,536) describes a measuring systemfor helicopter rotor blades, using both a moving light transmittingsource and a moving light receiving source. This system suffers from theneed to perform advanced calculations related to the exact location ofthe two light sources with respect to time, as well as the need tomaintain moving components in the system, which typically are more proneto failure than stationary components.

SUMMARY OF THE INVENTION

The present invention relates a system for measuring the size of anobject on a conveyor. The system includes a first triangulationrangefinder on a first side of the conveyor perpendicular to thedirection of travel of the conveyor. The first laser triangulationrangefinder is configured to shine a spot of light on one side of theobject, and to detect the distance from the first laser triangulationrangefinder to where the spot of light impinges on the one side of theobject. The system also includes a second laser triangulationrangefinder on a second side of the conveyor opposite the first side ofthe conveyor. The second laser triangulation rangefinder is configuredto shine a spot of light on an other side of the object, and to detectthe distance from the second laser triangulation rangefinder to wherethe spot of light impinges on the other side of the object. The systemfurther includes a light curtain array disposed in a plane normal to thedirection of travel of the object. The light curtain includes aplurality of light emitting elements and a plurality of light receivingelements respectively located on an opposite side of the conveyor. Thelight curtain is used to measure the height of the object as it passesthrough the light curtain array. The system also includes a lineartravel measuring circuit configured to determine the distance traveledby the object as it travels on the conveyor. The system also includes aprocessor configured to receive output from each of the above elements,and to determine a length, width and height of the object accordingly.

The present invention also relates to a method for determining a size ofan object on a conveyor, with the object having at least a first sideand a second side and a height, with the first side being opposite withrespect to the second side. The method includes the step of measuringthe distance from a first rangefinder to a first spot of light incidenton the first side of the object and the position of the object/conveyor,as indicated by the linear distance traveled. The method furtherincludes the step of repeating the above two steps until the first spotof light is no longer incident on the first side of the object for atleast a first amount of time. The method further includes the step ofcollecting an array of pairs of rangefinder distance and position fromthe above repeated steps. The method still further includes the step ofidentifying a first point in the array at which the first spot of lightwas initially incident on the first side of the object. The method alsoincludes the step of identifying a second point in the array at whichthe first spot of light was last incident on the first side of theobject. The method includes the step of identifying a third point in thearray at which the first spot of light was incident on the first side ofthe object at a location closest to a lateral side of the conveyor. Themethod also includes the step of determining a slope of a first lineconnecting the first point to the third point, and a slope of a secondline connecting the second point to the first point. Lastly, the methodincludes the step of determining the one side of the object to beelements in the array between the first point and the third point if theslope of the first line is greater than the slope of the second line,and determining the one side of the object to be elements in the arraybetween the second point and the third point if the slope of the secondline is greater than the slope of the first line.

BRIEF DESCRIPTION OF THE DRAWINGS

For a description of a method and apparatus according to the invention,reference is made to the following description and accompanyingdrawings, in which:

FIG. 1 illustrates the system according to the invention mounted on aconveyor;

FIG. 2 is a front view of the system according to the invention on aconveyor;

FIG. 3 is a front angle view showing the placement of the laserrangefinders used in the triangulation function for determining the sizeof an object according to the invention;

FIG. 4 gives a top view of the system according to the invention on aconveyor;

FIG. 5 gives a front angle view of an object on a conveyor traversingthe system according to the invention;

FIG. 6 gives a back angle view of an object on a conveyor traversing thesystem according to the invention;

FIG. 7 gives a back view of an object on a conveyor traversing thesystem according to the invention;

FIG. 8 is a block diagram of the digital computer, TPU and interfaceboard used in the system according to the invention;

FIG. 9 is a timing diagram illustrating the control of the scan linecameras used in the system according to the invention;

FIG. 10 is a schematic diagram for the rangefinder discriminator used inthe system according to the invention;

FIG. 11 is a timing diagram for the input signals to the light curtainfor determining the height of an object on the conveyor;

FIG. 12 is a timing diagram for the input signals for the pulsetachometer of the system according to the invention;

FIG. 13 is a high-level block diagram detailing the structure of thesoftware modules used in the system according to the invention;

FIG. 14a gives a profile of the detection of the spot of light on anobject over a period of time by the laser range finder triangulationsystem;

FIG. 14b shows the same profile of FIG. 14a, with a line correspondingto one side of the object computed as a linear least squares fit of asubset of points in the profile; and

FIGS. 15a-15d are plots of the arrays of data which are used to computecarton measurements.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A preferred embodiment of the invention will be described hereinbelowwith reference to the drawings. The system according to the inventionmeasures the length, width and height of objects, such as cartons, inmotion, as they are transported on a conveyor. The system does notrequire a break in the surface of the conveyor, nor does it requirealignment of the cartons prior to the measurement process. The system isable to handle measuring objects on very high speed conveyors with smallgaps between the objects.

The system includes a digital computer which receives and correlates thesignals received from four sensors mounted on and around the conveyor.These four sensors correspond to: 1) a first laser triangulationrangefinder on one side of the conveyor which measures the distance fromthe side of the conveyor to the side of the carton on the conveyor, 2) asecond laser triangulation rangefinder on the other side of theconveyor, 3) a light curtain array with horizontal beams running acrossthe conveyor in a plane perpendicular to the direction of travel of theobjects on the conveyor, which measures the highest point on the cartonpassing through the plane, and 4) a pulse tachometer which measures themotion (or speed) of the conveyor, and consequently measures the motionof the objects in the direction of travel of the conveyor.

In the preferred embodiment, the triangulation sensors are mounted in asingle plane perpendicular to the direction of travel of the objects onthe conveyor. The light curtain sensors are mounted in a separate planeoffset from that of the triangulation sensors. As an object passesthrough these "measuring planes", the digital computer collects threeprofiles: one for the top of the object, and one for each side of theconveyor. When the carton leaves the measuring plane of thetriangulation sensors, the digital computer interprets the side profilesin a manner described hereinbelow, in order to locate the four edges ofthe object. From these located edges, the computer readily calculatesthe length and width of the object. The top profile is used to calculatethe height of the object. From the height, length and widthdeterminations, the volume of the object is determined by amultiplication of these three values. This calculation assumes that theobjects are rectangular solids.

To prevent crosstalk, the laser frequency used in the triangulationmeasurements is different from that used in the light curtain; moreover,one or more sensors may have a filter element (not shown in the figuresfor clarity) for filtering out frequencies other that of its associatedlight source. Of course, if the respective measurement planes of thelight curtain array and the triangulation rangefinders are sufficientlyfar away from each other, then the light curtain sensor and thetriangulation rangefinders can each use the same frequency band withoutcausing substantial interference to each other.

The system 2 according to the invention, as mounted around a conveyor20, is shown in various angular views in FIGS. 1 through 7. In each ofthese figures, the system is mounted on a U-shaped frame 10 thattraverses the width of the conveyor 20. A first laser light source 30 ismounted on one vertical bar 12 of the frame 10, and a second laser lightsource 32 is mounted on the other vertical bar 14 of the frame 10. A topbar 16 connects the two vertical risers 12, 14, and it is situated abovethe conveyor 20 in a substantially transverse relationship with respectto the conveyor 20, as seen in FIGS. 1 through 7. Mounted below thefirst laser light source 30 is the control electronics 40, in which ishoused a digital computer (see FIG. 8) that computes the size of theobjects 60 on the conveyor 20 based on raw measurement data fed to it.Of course, other mountings of the system 2 around a conveyor 20 may beenvisioned by those of ordinary skill in the art, such as having a firstand a second riser on respective sides of the conveyor in which thefirst and second triangulation rangefinders are respectively situated,and not having a top bar connecting the two risers.

Referring now to FIGS. 1 through 7, the first camera 70 is situatedwhere the first vertical riser 12 and the top bar 16 meet, and a secondcamera 72 is situated where the second vertical riser 14 and the top bar16 meet. The first and second cameras 70, 72 are positioned such thatthe center of their respective coverage zones is the center of theconveyor 20. The first camera 70 picks up light from the first laserlight source 30 that reflects off of one side of the objects 60, and thesecond camera 72 picks up light from the second laser light source 32that reflects off of an opposite side of the objects 60. Each camera 70,72 has a field of view which encompasses the entire width of theconveyor 20, as seen in FIG. 2.

FIG. 1 shows a tachometer 80 and a tachometer mounting apparatus 85disposed on a cross member 90 below the top surface of the conveyor 20.The tachometer 80 is directly connected to the cross member 90, andcomprises a mechanical "trundle wheel" 87 of known diameter. Thistrundle wheel 87 is urged against the belt by tachometer mountingapparatus 85. The tachometer 80 measures the distance travelled of theconveyor 20, simply by counting revolutions (or fractions thereof) of"trundle wheel" 87 In the presently preferred embodiment, the trundlewheel is twelve inches in circumference, and hence each full revolutionof the wheel indicates another foot of linear travel by the conveyor 20.

The system 2 has the ability to continue to track the object 60 on theconveyor 20 by "dead reckoning", using the conveyor linear distancetravelled, as measured by the tachometer 80, may be used for trackingthe object 60 as it traverses along the conveyor 20. When the object 60reaches a configured transmission location, a serial message is sent toa host computer housed in the electronics 40. The serial messagecontains the dimensions and computed volume of the object 60, which canbe used for further processing of the object 60; e.g., sortation viavolume, weight, height, length or a combination thereof.

A block diagram of the digital computer (or processor) 800 used in thesystem according to the invention is shown in FIG. 8. In the preferredembodiment, the processor (host computer) 800 is a Motorola 68332microprocessor, with a CPU-32 core 805 and a Time Processing Unit (TPU)on-chip 810. Such a computer is described in detail in "MotorolaSemiconductor Technical Data Manual MC68332" (Motorola, 1993),incorporated herein by reference.

The processor 800 is complemented by an off-chip Random Access Memory(RAM) 820, an Erasable Programmable Read-Only Memory (EPROM) 830, and anErasable Programmable Read-Only Memory (EEPROM) 840. In the preferredembodiment, the Vesta Technology Inc. SBC-332 card is used, whichcontains the processor 800, the RAM bar co820, and the EPROM 830. Thecomputing and memory elements are interconnected by address/data busesindicated at 845 on FIG. 8.

The signal levels from the TPU 810 are all Transistor Type Logic (TTL)levels (0 to 5 volts DC), but of course other logic levels (e.g., CMOSor ECL) may be used without departing from the scope of the invention.Most of the sensors used in the system according the invention operateat other voltage levels, so an Interface Board (IB) 850 is used to holdthe level translators needed to bring all of the levels to a compatibleTTL level.

The TPU 810 allows rapid servicing of the sensor signals withoutintervention by the processor 800. The TPU 810 allows up to 16 channels,each of which can be configured to run a different micro-coded function.The TPU is described in detail in "TPU Time Processor Unit ReferenceManual" (Motorola, 1993), and in "Using the TPU Function Library and TPUEmulation Mode" (Motorola Semiconductor Programming Note TPUPN00/D,dated 1993) incorporated herein by reference. In the system according tothe invention, the following TPU functions are utilized:

1) SPWM--Synchronized Pulse Width Modulation. Each SPWM channelgenerates a pulse stream with a settable period and duty cycle. Severalchannels can be linked together to synchronize the phase betweendifferent channels. In the system according to the invention, thesechannels are used to generate the operating signals for the cameras usedin the laser triangulation rangefinders. Four of the sixteen availableTPU channels are configured as SPWM channels in the system according tothe invention. The SPWM mode is described in detail in "SynchronizedPulse-Width Modulation TPU Function (SPWM)" (Motorola SemiconductorProgramming Note TPUPN19/D, dated 1993) incorporated herein byreference.

2) ITC--Input Transition Capture. This TPU function counts inputtransitions and can generate interrupts after n pulses. In the systemaccording to the invention, these channels are used to capture thediscriminated video output of the camera and to count the beam arraypulses. Five of the sixteen available TPU channels are configured as ITCchannels in the system according to the invention. The ITC mode isdescribed in detail in "Input Capture/Input Transition Counter TPUFunction (ITC)" (Motorola Semiconductor Programming Note TPUPN16/D,dated 1993) incorporated herein by reference.

3) FQD--Fast Quadrature Decode. For this TPU function, two FQD channelscan be linked together to perform full quadrature signal decoding. Inthe system according to the invention, these channels are used to readsignals from the pulse tachometer. Two of the sixteen available TPUchannels are configured as FQD in the system according to the invention.The FQD mode is described in detail in "Fast Quadrature Decode TPUFunction (FQD)" (Motorola Semiconductor Programming Note TPUPN02/D,dated 1993) incorporated herein by reference.

4) DIO--Digital Input/Output. This TPU function allows level setting andtesting. In the system according to the invention, one of the sixteenavailable TPU channels is configured as a DIO channel, and it is used tofire a general purpose annunciation relay (SSR), as seen in FIG. 8. TheDIO mode is described in detail in "Discrete Input/Output TPU Function(DIO)" (Motorola Semiconductor Programming Note TPUPN18/D, dated 1993)incorporated herein by reference.

5) UART--Universal Asynchronous Receiver/Transmitter. For this TPUfunction, each channel can be configured to send or receive serial codeddata information. In the system according to the invention, two of thesixteen available TPU channels are configured as UART channels. One ofthe UART channels is used to transmit the raw dimensional data of theobject being measured to host computer 800, and the second of the UARTchannels is used to output the computed (by the host computer 800)measurement data of the object to an external source, such as a display(not shown in the figures). The UART mode is described in detail in"Asynchronous Serial Interface TPU Function (UART)" (MotorolaSemiconductor Programming Note TPUPN07/D, dated 1993) incorporatedherein by reference.

As stated earlier, the sides of the objects 60 on the conveyor 20 aremeasured using laser triangulation rangefinders. (Since the rangefindersare a known, fixed distance apart, subtracting the range from eachrangefinder from this known, fixed distance is seen to easily yield thedesired dimension information.) Of course, other types of opticaltriangulation rangefinders may alternatively be used, but laserrangefinders tend to yield more rapid, precise measurement readings. Inthe preferred embodiment, each side of an object 60 is measuredindependently with an associated laser source 30, 32 and a CCD camera70, 72 configured to detect light shined onto the object 60 by theassociated laser source, and thereby determine the distance of theobject 60 from an edge of the conveyor 20. Thus, each of the tworangefinders is used to measure a side of the object 60 (facing therespective rangefinder) independently of the other rangefinder. Thecorresponding light beams 36, 38 output from the light sources 30, 32are shown in FIG. 3.

On each side of the conveyor 20, light output from a laser sourcedefines a line in the measurement plane. The measurement plane 400 ofthe triangulation rangefinders can readily seen from the top view of thesystem, as seen in FIG. 4. The measurement plane 400 is horizontallypositioned and perpendicular to the direction of travel of the object60. The intersection in the measurement plane 400 of the triangulationrangefinder of the light output from the light source and the side ofthe carton facing the light source creates a bright spot of light 7 onthe object, as seen in FIG. 5.

In the preferred embodiment, this spot of light 7 is shined on theobject 60 about 1" above the surface of the conveyor 20. In this way,small-height objects on the conveyor 20 can be illuminated by the laserlight source 30, 32. If the spot of light 7 was even located closer tothe surface of the conveyor 20, then even smaller objects may beilluminated as they pass across the laser rangefinders, but thenreflections of light off the top surface of the conveyor 20 may occur,which would cause inaccurate readings by the respective line scancameras 70, 72 which receive the reflected light off of the object 60.Therefore, the location of the spot of light 7 about 1" above the topsurface of the conveyor 20 is a reasonable compromise between detectingthe sides of small-height objects, while maintaining a reasonabledistance from the top surface of the conveyor 20 to allow for properdetection of the spot of light 7 by the line scan cameras 70, 72.

The line scan cameras 70, 72 are positioned above the laser sources 30,32, as seen in FIG. 5. Each of the cameras 70, 72 has an array of manylight sensors (not shown in the figures) in a single line in the imageplane. Each camera 70, 72 is aimed at the center of the conveyor 20, andthe light sensors in each line scan camera 70, 72 are capable ofreceiving light from any point between the two edges of the conveyor 20.Each camera 70, 72 is aligned so that the illuminated spot 7 on the sideof the object 60 is imaged onto the array in the lens of the camera 70,72.

The discussion below will be given for a laser light source 30 and acamera 70 on one side of the conveyor 20, but it equally holds for thelaser light source 32 and the camera 72 on the other side of theconveyor 20.

The distance (a) from the laser light source 30 to the camera 70 is aknown value, since the laser light source 30 is fixed in position. Thereis a non-linear but monotonic relationship between which pixel on theline scan camera 70 is illuminated and the angle θ of the line from thespot 7 to the camera 70 from a vertical plane with respect to the camera70. Therefore, the range X from the laser light source 30 to the side ofthe object 60 can be determined by the following equation:

    X=a*tan (θ)

FIG. 5 shows the location of each of the values used in the aboveequation. The relationship between the pixel in the line scan camera 70illuminated by the laser spot 7 and the angle θ may be derivedanalytically or may be approximated by an interpolation from a table ofcalibration measurements. The latter approach is often preferable due tothe speed of lookups and interpolation compared to an extensivecalculation required by the former approach.

A visible solid state laser 30, 32 with a power level just under 1milliwatt is used in the preferred embodiment, but of course any amountof laser light that provides an acceptable amount of light that can beshined onto an object and thereby detected by a light detector may beutilized while keeping within the scope of the invention. A lineprojection laser may be substituted, for example, with the projectedlaser line set in a horizontal direction. The line projection laser isadvantageous in that it is much more tolerant of poor camera alignment.

The line scan camera 70, 72 used in the preferred embodiment is asolid-state charge-coupled device (CCD), and corresponds to the modelLC1912, manufactured by EG&G Reticon. Such a camera is described indetail in "LC1912 Series Camera, Operation and Instruction Manual(055-0320-MAN, January, 1993, Revision B), incorporated herein byreference. This camera 70, 72 has 2048 sensors on 13 μmeter centers, andcan be clocked at a rate up to 20 MHz.

The line scan camera 70, 72 used in the preferred embodiment receives aMaster Clock (MCLK) signal and a Line Transfer (LT) signal, as shown inFIG. 9. The master clock is derived from a crystal oscillator which alsofeeds the TCR2 input to the TPU. The line transfer signals are generatedusing the SPWM functions of the TPU referenced to TC2. The line scancameras 70, 72 used in the preferred embodiment requires that thesesignals be RS-422 level differential signals, and level shifters 856provided on the Interface Board (IB) in FIG. 8 perform thistransformation.

The MCLK signal determines the amount of time that each pixel will bepresented at the outputs. The LT signal initiates a transfer of chargefrom the light sensitive areas of the camera to charge-coupledbucket-brigade devices (i.e., capacitive devices) on either side of thelight sensitive areas. These areas are clocked out to an output port, ata rate of one pixel for each MCLK cycle. The MCLK signal and the LTsignal, as well as the LR-L and LR-R signal, are respectively outputfrom the host computer 800, through the TPU 810, through the InterfaceBoard (IB) 850, and then to the respective cameras 70, 72 to enable theclocking out of pixel data from the cameras 70, 72.

The Line Reset (LR-L, LR-R) signals allow for electronic shutter controlof the respective line scan cameras 70, 72 by throttling the integrationtime for the cameras 70, 72.

The outputs of each of the two line scan cameras 70, 72 used in thesystem according to the invention are two separate analog streams, onefor the odd pixels (ODD) 900 and one for the even pixels (EVEN) 910.This multiplexing of the respective outputs of the odd and even pixelsallows for a faster speed in clocking the pixel data to the outside tomore quickly prepare for the next light detection reading.

Analog discriminators 100 (shown in FIGS. 8 and 10) are set by adjustingthe trimming potentiometers POT1, POT2 to fire only on those pixelsilluminated by the laser spot 7. The output of the discriminator 100 iscalled the VID-L signal (for the discriminator 100 used with the leftcamera 70), and is detected by a TPU channel (see FIG. 8) configured toperform the ITC function described earlier. A separate discriminator isprovided for the right camera 72 for generating the VID-R signal.

As seen in FIG. 9, the photo-detected light from the third oddphoto-detector exceeds a predetermined threshold value Vt, whichindicates that the spot of light 7 incident on the object 60 has beenpicked up at a location on the conveyor 20 at which the third oddphoto-detector points to.

Referring now to FIG. 10, since the intensity of light detected by thethird odd photo-detector is greater than the predetermined thresholdlevel Vt, the VID-L pulse will be generated by comparator 102 (used withthe left camera 70) at a time corresponding to when the data is beingoutput by the third odd photodetector. Note that a potentiometer POT1,POT2 is respectively connected to a non-inverting terminal of the oddand even comparators 102, 104 that make up the discriminator 100. Withthis configuration, the predetermined threshold value Vt can be adjustedbased on various factors, such as background light, type of laser lightsource used, etc. The outputs of the comparators 102, 104 are wired-ORedat position 105 in FIG. 10.

Every time the line transfer (LT) signal fires (i.e., is active "low"),the current count of the MCLK signal is stored in a data buffer. In thepreferred embodiment, the LT signal fires at a 1 KHz rate (i.e., 1000times per second). In the preferred embodiment, data from the first oddphoto-detector is clocked out about 13 MCLK cycles after the firing ofthe LT signal.

When the VID-L signal trips the ITC function, the then-current MCLKcount is stored in a data buffer. The difference between these two MCLKcounts directly yields the "number" of the illuminated pixel. In FIG. 9,this number is five, and corresponds to the third "odd" pixel.

In the preferred embodiment, the first pixel that receives enough lightin order to cause the VID-L signal to be active "high" is determined tobe the pixel which is pointed at the precise location on the conveyorwhere the spot on the object is located. Of course, other procedures maybe utilized while keeping within the scope of the invention, such asobtaining a profile of all of the pixels, so that if more than one pixelcauses a VID-L signal to be output "high", a weighted midpoint of theseilluminated pixels may alternatively be used to estimate the position ofthe spot of light output from the laser light source.

The light curtain used in the system according to the invention measuresthe highest point of the object 60 as it passes through a perpendicularplane (with respect to the conveyor 20) defined by the light curtain.The light curtain 600 is shown pictorially as multiple horizontal laserlight beams between the respective vertical risers in FIG. 6. The lightcurtain 600 includes two arrays of photodevices: the beam-array emitter602 and the beam-array receiver 604. The emitter 602 has manyphoto-transmitters spaced apart at predetermined distances from eachother, such as 1/4 inch apart. The receiver 604 has just as manyphotoreceivers configured to receive light from a respective one of thephoto-transmitters, and so they are spaced apart at the samepredetermined distances as the photo-transmitters. In FIG. 6, each ofthe multiple horizontal lines of the light curtain 600 corresponds tothe location of a light beam from one of the photo-transmitters of theemitter array 602 sent in the direction of a respective one of thephoto-receivers of the receiver array 604.

In the preferred embodiment, the light curtain 600 is constructed usingthe BeamArray system, manufactured by Banner Engineering Corp., locatedin Minneapolis, Minn. Such a system is described in U.S. Pat. No.5,266,793, for example, incorporated herein by reference. When poweredand wired for continuous scanning using the internal clock of theBeamArray system, the beam arrays 602, 604 making up the light curtain600 generate the signals shown in FIG. 11. The CLK signal cycles eachtime that an emitter-receiver pair are enabled. The DATA signal cycleseach time that the enabled receiver is illuminated by the enabledtransmitter (and therefore is not blocked by an object). The RDY signalpulses every time the beam array has scanned to the end of the array andis starting at the beginning again.

These signals are at RS-232 levels for the Banner Engineering lightcurtain, and are converted to TTL levels by level shifters 880 shown onthe interface board (IB) 850 in FIG. 8.

The RDY signal, the CLK signal and the DATA signal are fed intorespective ITC channels on the TPU 810, as shown in FIG. 8. The CLK ITCchannel counts the number of emitter-receiver pairs sampled. The DATAITC channel counts the number of emitter-receiver pairs not blocked byan object passing across the light curtain. The RDY signal generates aninterrupt, which stores the difference between the CLK count and theDATA count as the height of the object in units of 1/4 inches, and thenzeros the two counters for the next height measurement.

The movement of the conveyor 20 is measured by a wheeled pulsetachometer 80 mounted under the conveyor 60 in contact with the returnbelt of the conveyor 60, as shown in FIG. 1. In the preferredembodiment, the RHQ-360 model, manufactured by Photocraft, Inc., locatedin Sandwich, Ill., is used as the tachometer. Using a 360-pulsetachometer 80 with 12 inch circumference wheels, each pulse indicatesthat the conveyor 20 has moved 1/30 inches. FIG. 12 shows the signalsutilized by the pulse tachometer 80. A first clock signal A and a secondclock signal B correspond to the movement of the wheel of the tachometer80. The count signal COUNT outputs a pulse for each rising and fallingedge of the first and second clock signals A, B. Each pulse of the countsignal corresponds to a 1/30 inch movement of the conveyor 60. The A andB clock signals output from the pulse tachometer 80 are fed into twolinked TPU channels running the FQD function, as shown in FIG. 8.

The system according to the invention also may be configured to providea weighing function for objects passing over a particular area of theconveyor. In the preferred embodiment, the apparatus (not shown) forproviding such a measurement is a Weigh-Bar system, manufactured byWeigh-Tronix, Inc., located in Armstrong, Iowa. The Weigh-Bar systemuses load cells. Such load cells are described in U.S. Pat. Nos.5,336,854 and 5,313,023, incorporated herein by reference. Of course,other types of weighing apparatuses may be used, and still remain withinthe scope of the invention.

Further, the system according to the invention is also configured toprovide a bar code scanning feature (see element 47 of FIG. 1) asobjects pass through a part of the conveyor. In the preferredembodiment, a bar code scanning system known as "SureScan" which ismanufactured by the LazerData Products Group of PSC Inc., of Webster,N.Y. is used to provide such a scanning of moving objects on a conveyor.The SureScan scanning system is provided along one or both sides of theconveyor, and can be placed close to the light curtain and/or laserlight source. The LazerData bar code scanning device is described in theSureScan Omnidirectional Reading System is described in detail in itsMay, 1994 manual which is available from LazerData and which isincorporated herein by reference. Of course, other types and makes ofscanning apparatuses may be used to scan bar codes off of movingobjects, and still remain within the scope of the invention.

The overall scheme for data acquisition and data flow in the systemaccording to the invention is shown in FIG. 13. For completeness, sourcecode for performing the functions described in FIG. 13 is provided as anappendix to the disclosure. In FIG. 13, there are three levels ofprocessing that take place in order to gather, tabulate and interpretdata to obtain the dimensions of the objects on the conveyor.

The first level is the microcode level 1300. The lowest levels of dataacquisition are performed by microcoded TPU functions without theintervention of the CPU. These functions are shown in the upper lefthand portion of FIG. 13.

The second level is the interrupt service routine level 1310. Each timethat a camera or beam array completes a scan of all of the sensors, aninterrupt service routine inserts the data into capture buffers 1340a,1340b, 1340c. The data from the capture buffers 1340a, 1340b, 1340c isthen used by the CPU to perform the required measurement computations.The interrupt service routines LT-ISR and RDY-ISR are shown in the topcenter portion of FIG. 13. The capture buffers 1340a, 1340b, 1340c areshown on the top right hand portion of FIG. 13.

The third level is the task level 1320. The operator interface,dimension calculations, and message transmission are performed by tasksthat are controlled by a cooperative round-robin multitasker. Thesetasks are shown in the bottom portion of FIG. 13.

The fundamental task of the two lowest levels of processing is to buildthree profiles of each object: the top, the left side and the rightside. As each object passes through the light curtain and laser rangefinder and triangulation circuit, the interrupt service routine LT-ISRinserts values for the left and right sides of the object into arespective left and right side capture buffer 1340a, 1340b, and theinterrupt service routine RDY-ISR inserts values for the top of theobject into a top side capture buffer 1340c. Thus, three arrays ofvalues corresponding to the left-side, right-side and top of the object60 are stored in their respective capture buffers 1340a, 1340b, 1340c.Each measurement in the arrays contains the raw size measurement and thetachometer count at the time the measurement was taken.

The RDY-ISR interrupt service routine also detects the trailing edge ofthe object leaving the array. When this trailing edge detection occurs,the RDY-ISR interrupt service routine copies the three object profilesfrom the capture buffers 1340a, 1340b, 1340c to the work buffers 1350a,1350b, 1350c. By this pipelined arrangement, the raw measurements of oneobject can be processed while the raw measurement data for the nextobject on the conveyor can be collected at the same time. The workbuffers 1350a, 1350b, 1350c allow for double buffering, so that data canbe taken for the next object on the conveyor (into the capture buffers1340a, 1340b, 1340c) while processing of the previous object is beingdone by the host computer from the data stored in the work buffers1350a, 1350b, 1350c.

The RDY-ISR interrupt service routine also sets a flag FLG to indicatethe arrival of new data to the SIZER task; i.e., data arriving from thework buffers 1350a, 1350b, 1350c. The SIZER task performs the dataevaluation, and queues the results into a SEND queue. In the preferredembodiment, when the object has completely moved past the light curtainarray, then each of the photodetector receivers will receive light fromtheir respective photodetector transmitters on the opposite side of theconveyor. This occurrence will trigger the flag FLG being set by theRDY-ISR interrupt service routine, to inform the host computer that thecurrent object is slightly downstream from the light curtain array. Thesetting of the flag FLG also allows the data in the capture buffers1340a, 1340b, 1340c to be transferred to the respective work buffers1350a, 1350b, 1350c for processing by the host computer, since no moreraw measurement data from the current object is forthcoming.

The REPORTER task monitors the queue and transmits the message to thehost computer when the tracking of the object on the conveyor indicatesthat the object has reached the designated transmit point. The point atwhich the object arrives at the designated transmit point can bedetermined by dead reckoning, in a manner similar to that previouslydescribed.

The SIZER tasks performs the data evaluation function, and it evaluatesthe dimension of an object from the data deposited into the work buffersby the lower levels of processing. It does so by performing thefollowing steps.

1.) Since cartons can be "skewed" (i.e., such that the carton sides arenot parallel with respect to the belt edge), it is possible to collectmeasurements from two sides of the object. A typical array, plotted withthe tachometer value (i.e, time) as the abscissa and the pixel number asthe ordinate, is shown as the array of points 1405 in FIG. 14a. TheSIZER task first selects the side of the object and eliminates the ends.This is done by the following procedure.

a) Identifying the cardinal points, which correspond to the first point(f), the last point (l) an the extreme closest point (e). These pointsare identified respectively by pixel-tachometer pairs (p_(f),t_(f)),(p_(l),t_(l)), and (p_(e),t_(e)).

b) Estimating the two slopes m₁ and m₂ using the following equation:##EQU1##

c) If |m₁ |>|m₂ |, then the points from point e to point l are used asdefining the side of the object. Otherwise, the points from point f topoint e are used as defining the side of the object.

2.) Once the side is determined, a subset of points is selected. In thepreferred embodiment, 10 evenly spaced points are selected between andincluding the endpoints. The raw values are converted to calibratedvalues, and a slope m and intercept b are estimated using anequal-weight linear least-squares approach. Such a line 1410 determinedby a linear least-squares approach is shown in FIG. 14b. By this step, aline defining the side of the object is determined.

3.) The end cardinal points are perpendicularly projected to the linearleast-squares fit line. The point at which the intersection occurs isdetermined to be an edge of the object. Of course, while in thepreferred embodiment, only the end cardinal point (i.e., point f orpoint l) is used to project a line to the least-squares fit line 1410, asecond linear least-squares fit line could also be obtained for theother (adjacent) side of the object. Then, the intersection of thelinear least-squares fit line 1410 defining a side of the object and thesecond linear least-squares fit line defining the other (adjacent) sideof the object would define an edge of the object. Alternatively, asingle laser camera pair may be used to identify three corners, which issufficient for rectangular solid objects, to determine the volume of theobject.

4.) Once the locations (xi, yi) of the four vertical edges aredetermined, the carton dimensions can be estimated by computing thedistance between adjacent edges using the distance formula:

    d=SQRT {(x1-x2).sup.2 +(y1-y2).sup.2 }

The length is estimated by computing the distance between edges on thesame side of the carton. The width is estimated by computing thedistance between edges on the same end of the carton.

As two estimates for each dimension are produced by the left and theright laser range finder systems, the largest value is chosen, sincemost measurement errors will underestimate the dimensions of an objectbeing measured. Again, other appropriate techniques may be alternativelyutilized, such as using the respective averages of the left side and theright side length and width estimates as the determined size of theobject.

5.) With the exception of the leading and trailing edges of the objectbeing measured, all of the top measurements should be commensurate,assuming that the objects on the conveyor are relatively rectangularsolids in shape. Consequently, after discarding the first few and lastfew height measurements to avoid edge effects, the height is estimatedby averaging all of the top measurements.

Therefore, with the above approach, a length, width and heightmeasurement of objects on a conveyor can be readily determined, nomatter the speed of the conveyor or the spacings between objects on theconveyor. If objects happen to overlap on the conveyor, the systemaccording to the invention will treat the combined shape of theoverlapped objects as one object for purposes of measurement.

Unlike conventional systems, since the length and width measurements arenot computed by a laser light curtain or similar device, the conveyorcan be a unitary device, and there is no need to have a break in thesurface of the conveyor in order to place photo-transmitters orphoto-receivers in their proper place.

In the system according to the invention, if an object is perfectlyperpendicular with respect to the left and right side light beamsemitted from the respective left side and right side laser lightsources, the algorithm as described above is slightly modified sinceeach of the points measured in the step a) correspond to the extremeclosest point e to the conveyor; i.e., all points including points f, e,and l on the same straight line. In this instance, one side of theobject is determined as the distance between the first point f and thelast point l. It is not known if this distance corresponds to the lengthor width of the object at this time. For ease of description, this sidewill be identified as side one. The dimensions of side one and itsopposite side are determined in the same manner.

A linear least-squares fit is computed for all of the points, and apoint on the linear least-squares fit closest to the first point f isdetermined to be one edge for side one of the object, and a point on thelinear least-squares curve closet to the last point l is determined tobe the other edge of side one of the object. As a result of thiscomputation finding the corners of the object, the dimensions of theobject can readily be determined, using step 4), described above.

If a single laser/camera pair is used, it is possible to determine threecorners of the object only if the object is skewed along the directionof the conveyor, i.e., not oriented with its sides parallel to thedirection of travel along the conveyor. Thus, using a singlelaser/camera pair is sufficient to measure the volume of all skewedobjects on the conveyor. Objects may be intentionally skewed on theconveyor to permit use of a single laser/camera pair for thelength/width measurements.

FIGS. 15a through 15d illustrate captured profiles of objects on aconveyor using the steps 1) through 5), given above. As readily seenfrom FIGS. 15a through 15d, once the lengths of the object are computed,the widths of the object can be determined by forming a line between theendpoints on each side of the computed lengths of the object.

While a preferred embodiment of the invention has been described herein,it will become apparent to those of ordinary skill in the art thatmodifications to the above-described invention may be performed whilestill keeping within the teachings of the invention. For example, othertypes of light detection cameras instead of a CCD camera may beutilized. Further, the system could operate at a non-visible frequency,such as an infrared wave band or an ultraviolet wave band, so that the(ultraviolet or infrared) light source and the camera would beconfigured to respectively output an electromagnetic wave at therequired wavelength.

What is claimed is:
 1. An apparatus for measuring the size of an objecton a conveyor, comprising:a first triangulation rangefinder configuredto shine a spot of light on one side of the object and to detect alocation where the spot of light impinges on the one side of the object,said first triangulation rangefinder configured to output a firstdistance signal indicative thereof; a second triangulation rangefinderconfigured to shine a spot of light on an other side of the object andto detect a location where the spot of light impinges on the other sideof the object, said second triangulation rangefinder configured tooutput a second distance signal indicative thereof; a light curtainarray configured to measure a height of the object and to output aheight signal indicative thereof; a conveyor travel measuring circuitconfigured to determine a current speed of the conveyor, and configuredto output a conveyor travel signal indicative thereof; and a processorconfigured to receive said first distance signal, said second distancesignal, said height signal, and said conveyor travel signal, and todetermine the size of the object in response thereto, wherein said lightcurtain array is positioned above said conveyor, wherein said conveyorhas a continuous surface in a region directly below said light curtainarray, and wherein said first and second triangulation rangefinders eachrespectively include: a fixed light source located on a respective sideof the conveyor and configured to output a light beam; and a fixedcamera located above the conveyor and on the respective side of theconveyor and configured to receive light off of the object due to thelight beam impinging the object, said fixed camera including a pluralityof light detecting elements for performing an electronic scan todetermine a location above the conveyor at which the light beam hits arespective side of the object, said fixed camera being fixed in positionand orientation with respect to the conveyor during the electronic scanof said fixed camera.
 2. The apparatus according to claim 1,wherein saidlight curtain array is disposed in a vertical plane, said light curtainarray including a plurality of light emitting elements and a pluralityof light receiving elements respectively located on an opposite side ofsaid conveyor, and wherein said light curtain array measures the heightof the object as the object passes between the light emitting elementsand the light receiving elements.
 3. The apparatus according to claim 1,wherein said conveyor includes a first side, a second side opposite thefirst side, a third side and a fourth side opposite the third side, adirection of travel of said conveyor is from the first side to thesecond side, and the third and fourth sides form respective edges ofsaid conveyor,and wherein said first triangulation rangefinder isdisposed against the third side and said second triangulationrangefinder is disposed against the fourth side.
 4. The apparatusaccording to claim 1, wherein said fixed light source for each of saidfirst and second triangulation rangefinders are laser light sources. 5.The apparatus according to claim 1, wherein said fixed camera for eachof said first and second triangulation rangefinders comprise line scancameras.
 6. The apparatus according to claim 1, further comprising:aweight measuring device disposed under a top surface of the conveyor andconfigured to determine a weight of the object as it passes over theweight measuring device.
 7. The apparatus according to claim 1, furthercomprising:a bar code determining device disposed on one side of theconveyor, and configured to read a bar code on said object as it passesacross said bar code determining device.
 8. The apparatus according toclaim 1, wherein said first and second triangulation rangefinders andsaid light curtain array are each mounted on a U-shaped frame, saidU-shaped frame having a first vertical riser situated near one side ofthe conveyor, a second vertical riser situated near an opposite side ofthe conveyor, and a horizontal member connecting said first and secondvertical risers and disposed above said conveyor.
 9. The apparatusaccording to claim 8, wherein said light curtain array comprises aplurality of light emitting elements and a plurality of light receivingelements,and wherein said light emitting elements are disposed on saidfirst vertical riser and light receiving elements are disposed on saidsecond vertical riser.
 10. The apparatus according to claim 1, whereinthe processor determines the sides of the object by a least-squaresestimation process.
 11. A method for determining a size of an object ona conveyor, said object having at least one side and an adjacent sideand a height, comprising the steps of:a) identifying a location at whicha spot of light is incident on either the one side or the adjacent sideof the object; b) determining a time the identification is made in thestep a); c) repeating the steps a) and b) until the spot of light is nolonger incident on either the one side or the adjacent side of theobject; d) measuring a speed of the conveyor at the time theidentification is made; e) collecting an array of locations from therepeated steps a) and b), the array being categorized according to timeof identification of each of the locations in the array; f) identifyinga first point in the array at which the spot of light was initiallyincident on either the one side or the adjacent side of the object; g)identifying a second point in the array at which the spot of light waslast incident on either the one side or the adjacent side of the object;h) identifying a third point in the array corresponding to a minimumdistance; i) determining the location of three corners of the object byutilizing at least the first, second and third points; j) determiningthe length of the one side and the adjacent side using the locations ofsaid three corners, the associated time for each location and the speedof said conveyor; k) determining the height of the object whiletraveling on said conveyor; and l) determining the size of the objectfrom the length of the one Bide, the length of the adjacent side, andthe height of the object.
 12. The method according to claim 11, furthercomprising the steps of:m) forming a line using a linear least squaresfit based on the locations in the array between and including one of thefirst and third points and the second and third points; and n)determining an intersection between the line formed in the step m) andone of the first and second points, said determined intersectioncorresponding to a corner of the object.
 13. The method according toclaim 12, further comprising the steps of:o) determining a weight of theobject on the conveyor.
 14. An apparatus for measuring the size of anobject on a conveyor, comprising:first triangulation rangefinder meansfor shining a spot of light on one side of the object and for detectinga location where the spot of light impinges on the one side of theobject, said first triangulation rangefinder means outputting a firstdistance signal indicative thereof; second triangulation rangefindermeans for shining a spot of light on an other side of the object and fordetecting a location where the spot of light impinges on the other sideof the object, said second triangulation rangefinder means outputting asecond distance signal indicative thereof; light curtain array means formeasuring a height of the object and outputting a height signalindicative thereof; conveyor distance traveling means for determining adistance traveled by the conveyor, and outputting a conveyor distancetraveling signal indicative thereof; and processing means for receivingsaid first distance signal, said second distance signal, said heightsignal, and said conveyor distance traveling signal, and for determiningthe size of the object in response thereto, wherein said light curtainarray means is positioned above said conveyor, wherein said conveyor hasa continuous surface in a region directly below said light curtain arraymeans, and wherein said first and second triangulation rangefinder meanseach respectively include: a fixed light source located on a respectiveside of the conveyor and configured to output a light beam; and a fixedcamera located above the conveyor and on the respective side of theconveyor and configured to receive light off of the object due to thelight beam impinging the object, said fixed camera including a pluralityof light detecting elements for performing an electronic scan todetermine a location above the conveyor at which the light beam hits arespective side of the object, said fixed camera being fixed in positionand orientation with respect to the conveyor during the electronic scanof said fixed camera.
 15. The apparatus according to claim 14, whereinsaid conveyor includes a first side, a second side opposite the firstside, a third side and a fourth side opposite the third side, adirection of travel of said conveyor is from the first side to thesecond side, and the third and fourth sides form respective edges ofsaid conveyor,and wherein said first triangulation rangefinder means isdisposed against the third side and said second triangulationrangefinder means is disposed against the fourth side.
 16. The apparatusaccording to claim 15, wherein said first and second triangulationrangefinder means each respectively include:a fixed light source locatedon a respective side of the conveyor and configured to output a lightbeam; and a fixed camera located above the conveyor and on therespective side of the conveyor and configured to receive lightreflected off of the object due to the light beam impinging the object,said fixed camera including a plurality of light detecting elements fordetermining a location on the conveyor at which the light beam hits arespective side of the object.
 17. The apparatus according to claim 16,wherein said fixed light source for each of said first and secondtriangulation rangefinder means comprise laser light sources.
 18. Theapparatus according to claim 8, wherein said fixed light source of saidfirst triangulation rangefinder is disposed on said first verticalriser, said fixed light source of said second triangulation rangefinderis disposed on said second vertical riser, said fixed camera of saidfirst triangulation rangefinder is disposed at a position where saidfirst vertical riser meets said horizontal member, and said fixed cameraof said second triangulation rangefinder is disposed at a position wheresaid second vertical riser meets said horizontal member.
 19. Anapparatus for determining at least one of a size and a relative positionof an object on a conveyor, comprising:a first fixed light sourcedisposed on a first side of the conveyor and configured to shine a firstspot of light above the conveyor at a predetermined height above theconveyor; a first fixed camera disposed above the conveyor andconfigured to determine where the first spot of light impinges on theobject as the object crosses a first particular location above theconveyor, said first fixed camera outputting a first distance signalindicative thereof, said first fixed camera performing an electronicscan over the first particular location of the conveyor, said firstfixed camera being fixed in position and orientation with respect to theconveyor during the electronic scan of said first fixed camera; a secondfixed light source disposed on a second side of the conveyor oppositethe first side and configured to shine a second spot of light above theconveyor at the predetermined height above the conveyor; a second fixedcamera disposed above the conveyor and configured to determine where thesecond spot of light impinges on the object as the object crosses asecond particular location above the conveyor, said second fixed cameraoutputting a second distance signal indicative thereof, said secondfixed camera performing an electronic scan over the second particularlocation above the conveyor, said second fixed camera being fixed inposition and orientation with respect to the conveyor during theelectronic scan of said second fixed camera; a conveyor travel measuringcircuit configured to determine a current speed of the conveyor, andconfigured to output a conveyor travel signal indicative thereof; and aprocessor configured to receive said first distance signal, said seconddistance signal, and said conveyor travel signal, and to determine saidat least one of the size and relative position of the object in responsethereto.
 20. A method for determining characteristics of one or moreobjects on a conveyor, where each of said one or more objects has asubstantially rectangular shape, comprising the steps of:a) identifyinga location at which a spot of light is incident on the one or moreobjects; b) determining a time the identification is made in the stepa); c) repeating the steps a) and b) until the spot of light is nolonger incident on the one or more objects; d) collecting an array oflocations and times the identifications were made from the repeatedsteps a) and b); e) from the collected array of locations and timesobtained during the step d), determining circumferential characteristicsof the one or more objects; f) from the circumferential characteristicsof the one or more objects obtained from the step e), determiningwhether the one or more objects actually corresponds to one object ormore than one object; and g) outputting a signal based on thedetermination made in the step f).